Ontology-Based Annotation of Paintings Using Transductive Inference Framework

Domain-specific knowledge of paintings defines a wide range of concepts for annotation and flexible retrieval of paintings. In this work, we employ the ontology of artistic concepts that includes visual (or atomic) concepts at the intermediate level and high-level concepts at the application level. Visual-level concepts include artistic color and brushwork concepts that serve as cues for annotating high-level concepts such as the art periods for paintings. To assign artistic color concepts, we utilize inductive inference method based on probabilistic SVM classification. For brushwork annotation, we employ previously developed transductive inference framework that utilizes multi-expert approach, where individual experts implement transductive inference by exploiting both labeled and unlabelled data. In this paper, we combine the color and brushwork concepts with low-level features and utilize a modification of the transductive inference framework to annotate art period concepts to the paintings collection. Our experiments on annotating art period concepts demonstrate that: a) the use of visual-level concepts significantly improves the accuracy as compared to using low-level features only; and b) the proposed framework out-performs the conventional baseline method.